Regional Educational 


Institute of STUDY SNAPSHOT Laboratory 
Education Sciences July 2020 Mid-Atlantic 


At Mathematica Policy Research 


Using Data from Schools and Child Welfare Agencies to 
Predict Near-Term Academic Risks 


Pittsburgh Public Schools (PPS), the Propel Schools charter network, and the Allegheny County Department of 
Human Services (DHS) want to better identify students at risk for academic problems in the near term. The stake- 
holders partnered with the Regional Education Laboratory Mid-Atlantic to develop an approach for identifying 
at-risk students using school data linked with data on child welfare events, justice system involvement, and other 
human services involvement and public benefits receipt. Identifying the students most likely to have certain types 
of academic problems in the near term can support an early warning system that allows educators and human 
services staff to better target resources and lessen risks before they become more serious. 


The study included students in all grades in PPS and Propel Schools in the 2015/16 and 2016/17 school years. Pre- 
dictive models used linked school and Allegheny County DHS data to generate risk scores indicating the probability 
that each student would experience certain academic outcomes: chronic absenteeism, out-of-school suspensions, 
failure of a core subject course, term grade point average (GPA) below 2.2, or a score below the basic level on state 
tests in the upcoming school period. Accuracy of predictions was examined for each outcome using school data 
only and combined school and Allegheny County DHS data (out-of-school data) to make predictions. Models pre- 
dicting outcomes in PPS identified at-risk students at a level that is generally considered strong in social science." 
Most models for Propel Schools also met this threshold, despite the smaller sample sizes. 


Key findings 

¢ Prior academic problems are strongly related to future academic problems. Students who experience one of the 
academic problems examined in this study are substantially more likely than other students to experience that 
problem again in the next term or year. For example, PPS high school students who had a low term GPA in the 
prior period were 64 percentage points more likely to have a low GPA in the following period than students who 
did not have a low GPA in the prior period. 

¢ Some out-of-school variables, as reflected in human services data, are strongly related to students’ future aca- 
demic problems. For example, PPS high school students who started an out-of-home placement were 45 per- 
centage points more likely to be chronically absent in the next quarter than students who did not. Emergency 
shelter stays of seven days or more and active juvenile justice cases also have strong relationships with aca- 
demic problems in the following months. Educators and human services staff may be able to use this informa- 
tion to understand and address underlying challenges in students’ home lives and reduce the related academic 
problems. 

e Accuracy of predictions is better for course and test performance outcomes. The available predictor data are 
better able to identify students who will have a low GPA, course failures, or scores below the basic level on 
state tests. The predictor data are not as effective at identifying students at risk for chronic absenteeism and 
suspensions. Those outcomes are likely more affected by random events or events not captured by available 
school and human services data systems. 
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